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ABSIBICT 

This^ report illustrates thai aeasores of the 
effectiveness of teachers* iapleaentation of a program can be derived 
from the instructions given to teachers on how to ase the cnrricnlna, 
rhat such neasores can be ased to improve the effectiveness of 
teacher training, and that better teacher training can be shown to 
lead to better learning on the part of students. By studying the 
relationships between behavior of teachers vithiii curriculum pro^raa^ 
and student outcomes, curriculum developers, publishe^rs, and schopl 
personnel can have a significant impact on the development and 
assessment of teacher education programs and on modification of ti^e 
curriculum materials themselves. Discussion 4-8 divided into four 
sections: a brief review of the research on teacher behavior within 
curriculum programs; a case study^of the Dist'ar instructional 
program, which illustrates how observation^ systems and measures of 
student gain can be used to establish relationships between 
instructional activities and pu^il outcomes; suggestions for future 
research in curriculum programs; and implications of this 
investigation fcr pre.service and inservice teacher education 
programs. (KS) 
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Substantial resources are devoted each year to' the In-servlce training 

♦ ■ 

of teachers to use curriculum materials in ways prescribed by the curriculum 
developers. Yet, the extent to which the teachers implement the currlculim 
is often not measured. Still less often does anyone determine if students 
learn more when the teachers follow the developers' instructions. ^* • 
The first purpose of this chapter is to illustrate by means of a cMe 



study that: 



r 



measures of how well teachers are implementing a 
curriculum can be derived from clear^ and detailed 
instructions to teachers on how to use the curriculum; 

— such measures can be used to determine and Improve 
the effectiveness of teacher training; 

— better teachet training can lead, and can be shown » 
to leady to bettet^iatudent lemming. 

The second purpose is to argue that curriculum developers , publishers, 
and school personnel — by studying the relationships' between teacher behaviors 
within curriculum programs and student outcomes — can have a significant impact 
on the development and assessment of teacher education programs and on the^ 

4 

modification of the curricultim materials themselves. - ^ 

This chapter is divided into four sections: (1) a brief review of the 
research on teacher behavior within curriculum programs; (2) a case study ^ 
from a particular curricultmi package, illustsating how observational systems 

. 'fi ■ 

-The author would like to acknowledge John B. Gilpin and A. Lynn Hisselt, 
University of Illinois, and Linda Siegel for criticizing earlier drafts of 
this chapter. 



and measures of student gain can be used to establish relationships between , 
instructional ac1:ivities and pupil 'outcomes; (3) suggestions for future 
research in curriculum programs; and (4) implications of this research for 
teacher pre*service and in-service education programs. * 

' ■ ■ / 

A Summary of the Research 

Curriculum research on teacher behaviors has focused on two types of 
instructional activities. First, there are those specific behaviors which 
are believed to be important for the success of a given program. Many of 
these prescriptions for behavior are found in curriculum guides, workshop 
tkianuals, and the vfritings of program authors. Second, there are those be*, 
haviors which are believed to be important for the. success of a wide range 
of programs — that is, across all or most curricula or teaching situations. * 
Teacher clarity, flexibility, enthusiasm, and use of student ideas are examr- 
plea of program-general behaviors which have been studied (Rosenshine & 
Furst, 1971). 

I 

Program^specif 1^ and program-general teacher behaviors are not always 
mutually exclusive. That la, some behaviors which are specific to a parti-* 
cular curriculum may also be Important for other programs as well. The 
^"(listinction between these two types of instructional behaviors resides in the 
nature of the research. Curriculum-general studies examine those behaviors 
wt^ich may be important for ^the implementation and success of a wide range of 
curricula. Curriculum-specific studies^ hov^er, examine those behaviors 
which may be impottant for the implementation and success of a particular 
curriculum — irrespective of its importance fbr other curricula. 



Curriculum-Specific Studies 

Studies which have used observational systems to describe instructional 
activities considered important for the implementation of a curriculum pro* 
gram, have found vide variation between classrooms using the same program 
(e.g.; Bissell, 1971; Gallagher, 1966, 1968; Katz, 1968; Lindvall & Cox, 
1970; Niedermeyer & Dalrymple, 1970). * For example, Gallagher (1966) studied 
the instructional behaviors of six teal:hers using the same unit from the 
Biolo^cal Sciences Curriculum^ Study (BSCS) program. The teachers had some 
preyious BSCS training and their students had been selected on the basis of 
high ability. Three cotfsecutive class sessions were recorded during the 
introduction of photosynthesis and analyzed by Aschner and Gallagher's topic 
classification system. 

On almost all measures of teacher behavior there were significant dif* . 
ferences among the six teachers. For example, one of the functions of class 
discussions, according to the BSCS develqo^is, is to allow the student the 
opportunity t^ clarify and enrich his understanding of new concepts.' In 
general, however, t'he teachers tended to speak three to four times as much ^ 
as their students. When the focus of a topic in a class session was on 
description (defining or describing aspects of a concept or event), the amount 
of teacher talk ranged from 76Z to 9'7Z; when the focus was on expansion (leading 
the group off to other lines of thinking or encouraging hew associations) , the 
amount of teacher talk ranged from 67Z to 100%; and when the focus was on 
explanation (focusing on reasoned a^ument through sequential deductive steps 
of thinking), the amount of teacher talk ranged from 59% to 91%. 



Regretably, the investigator. dkld not relate this variatli5h in teacher 
behavior to changes iti student learning or attitude. , For example, what are 
the relationships between the amotinq of teacher talk and the student's in-* 
creased ability to understand abstract biological concepts? - Does an increase 
in the teacher's inqulry*strategy behaviors which are intended by the BSCS 
curriculum designers enhance or suppress student achievement, or is the effect 
negligible? Given a behavior that affects academic achievement, what are the ^ 

r •■ ■ r _ • 

concommitant effects in the pupil's attitude toward the study of biology, 
toward the teacher and school, or toward himself? These qyestions (and 
others) will have to be stiKlied before effective training program? for BSCS 
teachers can be designed. 

Curriculum*General Studies 



While curriculum studies focusing on general instructonal activities 
attempted to relate these behaviors to gains in student outcomes, the ob- y 
servational instrtmients used were designed to apply to all types of programs 
and educational settings (e.g., Flanders, 1970; La Shier & Westmeyer, 1967; 
Soar, 1971; Soar '& Soar, 1972; Soar, Soar, & Ragosta, 1971; Walberg, 1969). 
For exaiiq)le. Soar and Soar (19j72) monitored eight classrooim^n each of seven 
Follow-*Th rough programs along\wlth two comparison classrc^ms for each program* 
One day was spent in each classroom by a team of two observers.' Instead %)f 
developing program-specific observation instruments, ^oat aiid Soar used four 
general observational systems: the Florida Affective Categories (FLAC), 
a measure of "nonverbal expression of affect in the ^assroom, and the extent 
to which J^ndividual pupils or small groups of jprupils [are] cendral in 



classroom activities" (Soar & Soar, 1972, p, 234); the Teacher Practices 
Observation Record (TPOR) , a measure of "the consonance of a teacher *s prac- 
tices with John Dewey's 'experlisentalism' [and] teacher behavior that 
widely practiced in the public schools" (So^r & Soar, 1972, p, 234) ; the 
Florida Taxonomy of Cognitive Behavior, a measure o*f different levels of 
intellecjdual activity such as memory, translation, interpretation, application. 



analysier^ S3mthesis, and evaluation; and Ober's Reciprocal Category System, 
an expansion of the Flander's Systi^m and a measure of teacher and' pupil talk. 

The, investigators correlated the ratings and counts (factor scores de-|f^ 
rived from the four observation' Instruments) with measures of gain in student 
learning (class mean residual gain). For example. Soar and^oar found that 
the factor representing settings in whfth pupils are working in small groups 
or as individuals, work on a complex^ task is s^t by the teacher without con- 
tinuing direction. • . , or pupil behavior or work has occasional close 



direction" (Soa;- & Soar. 1972, p. 246) correlated positively ( r = .55) with 
gains in knowledge and of abstract subject matter. 

It Is plausible, however, that the most critical variables ^hich affecte 
pupil gains were not includ^jd in the general observational instruments. The 
ability to follow ft pre-spe^fled format without even minor deviations and to 
reqijlre unison groUp responses may be flnportaht variables in the Bngelmann* 



:iex 



Becfer Follow-Through Model; whereas in the Educational Development Corpora- 
tion (EDC) Program,^ the ability of the teach respond to the needs and 
interests of the chlldren^o as to form a rich and stimulatilng environment, 
may be essential to the realization of th^ program's goals and objectives. 
Yet, a general observation insttumeht is likely to^be insensitive^ to these 

' ^ »v ... 
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student learning. I t;^t tempts to ^orrect the deficiencies of eiarlier curri- 



program-specific variables Therefore, in •addition to observational instru- 
ments which reflect program-general or "acrdss the board" instructional > 
activities, development of observational measures which /reflect those teacher 
and pupil behaviors most emphasized by the ciu^culum designers seems useful. 

A Proposed Research Strategy 

The following "descriptive-correlational-experlmental-loop" research 
^trategy (Rosenshine & Furst, 1973; Siegel, 1972, lSf74) is a model for -a^rste- 
matically approaching optimal teacher training programs, thereby influencing 

culum research on teacner behaviors and is used as the basis- for^th^ case 
study presented in the pecon3 part of this chapter. The model has five phases: 

1. Train a group of teachers to use the package of curriculum materials 
according to the authors* specifications. Most large-scale curriculum pro- 
jects have developed materiasls which can be used for in-service training. 
Also, a careful .examination of teacher guides -of ten suggests guidelines for 
instructional /^ehavior — presenting theSnaterials^, arranging^ the classroom 
environment, secjuencing the lesspns, reacting to 3t||dent ep?o«, and so on. 

2. Develop and use obsefv^lfcrtlarsystems^p describe the instructional 
variables which are considered spec|.fic to the program and most emphasized 



by thp curriculum planners, as well as the variables which* are considered 
to(|have general educational importance (and which may or may not be emphasized 
by the curriculum designers). * 

'3. Studl^ the relationships between instructional activifies en^ behavfc^al 
change in the students in a variety of outcomes (e.g., cognitive, affective). 
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At least tb& following ten questions 8hotild be> asked: 

a. Tp What Extent were the Instructional activities within the 
program those which were Intended by the curriculum developers?;. 

b. ' Did the classrooms (br other unljis^ within the program differ 

In their, lose of Instructional activities specific to the pro* 
gram? ^ • 

\ ^ ■ - 

c. Did the .classrooms within the program differ In the use of 

' geifer^L Inst^ructlona^ activities considered Important for student 
J. ' growth? * V 

d. Were the classrooms within the program different on the outcpme 
sfeasures of Interest? ^ 

e. VQiat was the relat!^pshlps between use of program-spQidlf Ic 
actlv^ftles and stud^t growth? ^ - 

• ff 

f . -What was the relatlo^ishlp between general instructional activi- 

ties and student growth? 

g. Were there differences in student ^growth among classrooms of 
teachers who were high, average, or below average in their 

fidelity to^ the intentions of the curriculum developers? 

. • • «^ ^ ^ r • * 

h. Were there differences in student growth among classrooms of 
teachers who were high^ aver^'e, or below average in their 
use of general instruotional activities? 



1. Were cla^Sfooms which^ere high, average, or J^elow average in 
student growth different in their fidelity to the 4ntentions of 
theiycurriculum^developers? ^ ^ 

j. Were classroo^ns which were high, average, or below average in 
student growth different, in their use of general ins^^ctional 
activities ?l/tR6senshine, 1971, p. 84) -"^^ 



Averaging implementation ratings across visits ^or those teache^ whose 
ratings increase, decrease, or ^fe erratic throughout the year (e.g^, Ibw, 
average, high; high, ayerage , low; high^\low, average; high, low, high) may 
be misleading. ^i)escrlbiii^ the teachers as medium or average Implementors is 
not as. descrilptive as perhap6^"ascendant '^descendant," and/or ^'erratic.** 
These patterns could also occ^ir in student behavior if measures of student 
outcoimes are taken at different intervals throughout the year.v 



4. Modify the training procedures, observation instruments, and/or 
curriculum materials on the basis o¥ the correlational and quasl*experlmehtal 
studies completed in phrase three. iPerhaps some variables (e.g., a particular 
subset of teacher "behaviors) aij^e ^mDre predictive of changes in student out* « 
comes than other variables. Perhaps certain variables suggest that sQme of 
the basic assumptions of the training programs or curriculum materials are ifk^ ' 
doubt. " • 

5. Retrain some or all of the teachers. ^Condtlit new stuHles to ^determine 
the effects of t^e ^modifications and to determine the new relal:ionships be* 
tween instructional activities and Student learning. By recycling' throygh 
phases one through , four, ^ the curr^^lum designer, publisher, and .researcher 
successively"^ approximate optimum training procedures and curriculum materials, 
X\iw affecting gains In student achievement or ^ther measures of interest. 

Further problems and suggestions for developing measures of Instructional 
activities and student outcomes, designing research studies, and analyzing 
data are presented elsewhere (sep, for example, Flanders, 1970; Gage, 1969; 
Medley & Mltzel, 1963; Rosenshine, 1970, 1971; Rosenshine & Furst, 1971, 1973; 
Tdtsuoka, 1972). 

, An Example of Research on Teacher Behayiors Within a Curriculum Program 

One "pass" through the above "descriptive-correlational-experime^ajr^-^loop" 
research paradigm has been completed, with the Dis tar ^Instructional System. 
TH^vfocus 1^^ this section will be on describing this curriculiji^ package, 
illustrating the development of the program*specif ic observation system used 
in the research, reporting the summary findings, and ciescribing'implicat^ilms) 
of this research for training Distar teachers . - 



Description Jof"^tRe Dlstar Instructional System 

One of the nkmt successful (Mct)anlels, 1975; Becker & Engelmann, 1974; 



Office of Education, 1974; Stalllngs & Kaskowltz, 1974; Becker &. Sngelm^mi, 

^^^73; Science Research Associates, 1971a) ^and controversial of all the early 

childhood and primary grade curriculum materials programs is the Distar 

Reading, Language, and, Arithmetic proj^rams (Engelmann & Bruner, 1969, 1970, 

1974; Engelmann & Camine, 1969,^1970, 1972,^ 1974; Engelmann 6 Osbom, 1970, 

1972; Engelmann, Osbom, & Engelmann, 1969; Engelmann & Steams, 1972). These 

programs are the principal materials used in the Engelmann*Becker Follow^ 
2 

Through Model. Unlike other^ programmed materials, the Distar program is not 
a self-'lnstructional program^ Instead, the teacher follows a carefully strtic- 

' V ° 

tured and logically sequenced teaching program. The presentation Hooks, 
provide the teacher with a script — a series of demons tr«ft ions and tasks^-to 

feactjer-s role thus change^ from one of da-, 
^ ^ cliing a particular format to criterion 

(mastery), involving all of the children in the instruction, correcting mis- 
akes, providing feedback, and r^fn^rcing the children's responses. , ' 

A typicar|^first-grade clas^sroom i^ divided into three grbups of children, 
"hom^eneous'*' by achievement, with |he daily^lesson being presented to each 
gxoixp ^separa^^^. A^ thirty-minute lesson consists of ^a series of group and 
individual tasks, or activities. Once the teacher obtains Jthe children^ 



For a more complete descriptrlon of the philosopfty <and methods used in the 
Engelmann-Beckey^Progr^, the Reader is referred to Engelmann (1969a, 1969b). 
and Maccoby and^Zellner (1^70). 
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attention, she proceeds with the first task, followlngthe format a& written 
In the presentation book. The students respond. The teacher then evaluates 
their answers: Inappropriate responses are corrected according to a pre- 
^specified correct lo;i paradigm and appropriate . res^otises are praised. After 
all of the tasks In the litsson have been presented ^In thl^^mansex^^he-teachfei^ 

presets relnjfbrc'ement mterial In the^ofm^f '^take-hJtes." The children's 

» ■ 

perfo^rmance on the "take-homes" also provides feedback to the teacher and ^ 

* I . . ' . ... 

parent . ' J)uring the next session, the group moves on to the tasks ±p the 

following lesson. . " 

^ An example of a lesson format app^ar^ In Figure X. t^sk is one of 

ny in Dlstai^eadlng I (Engelmanjp^HB^n^r , 1974). Its purpose, along with 
other formats. Is to teach the chijldren to sound auk a word, say it fast, 
and identify the word. I ' ^ 

> ^ ^ ■ \ . ^ 

Insert Figure 1 about here 

_ A ^ ■ ..... . ■ : 

Teacher Implementation Variables « ^ 

Since the first ^Distar program was published by Science Research Assq- ' > 
elates in 1969, certain basic assumptions as to how the teacher should behave' 
when implementing the curriculum materials Kkve been stlated explicitly^ Fiye > 
areas of teacher behavior are emphasized throughout teacher guides and 
training manuals. ♦(Notice that each area is a composite 6 f .several variables^ ^ 
and l^hould not be^ thought of as a single variable.) ^ ' 
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FolloicLng the For—t 

Thtt pictuTM md.taalu in the Distar Program are not designed 
to provide you [the teacher] vltb points- of departure for dls- 
cussloniL They are designed to achieve very specific objec- 
vtlves'. ^hese objectives vUl not be set If you talk too vuch» 
If 'you allow the children to sake koo May eicCraiii^ous observe- 
tfons, or If you depart fr6n the t|»k as It Is- specified in 
the progran. 

Usf the exact wording provided in the saterlaie, and do not 
fliake additional atatefents pr ^slypiltional questions unless 

tirchr 



the forvat calls for then. Let tnr children know that you 
are on the task. %iscourage Irrelevant, obeervat lone. (Engel- 
> aaon & Osbom^ 1970 » p. 12) 

B. Slgnala ^ 

Use clear signals for the children to respond,^ so that ||ey 
all reapond at the sam tiae. The children aren't perf#idJig 
acceptably unless kll bf them respond appropriately to every ^ 
question. If some do not respond to a question, the group's 
response is unacceptable. In such a situation^ some children 
may be learning to llaten to what others 4ay and imitate 
their responsee. . . . With clear signals, you will be able 
to get much more accurate feedback, from the performance of 
I the different children in She group. (Engelmann & Gamine » 1972 
p. 14) ^ 

C. Corrections and Criterion Teaching 

Correct only the part of the exerclae the child trouble 
with. Correct the mistake ionediately aft«r it occura. 
After correcting the child on the part of the task he missed, 
alwaya return to the beginning of the exercise and repeat the - 
exerclae. The reason for this procedure is that the children 
must learn to see each exercise as a series of stepe. The 
steps do not occur in isolation. They are related to a goal 
and to certain rules. 

Unless you always repeat a task from t^e beglqning and do not 
condlude that the children have been Corrected until they can 
go through the entire exerclae without mailing a mistake, the 
children may learn to handle each of the steps without ever ' 
seeing how the steps fit goethher in a pattern. 
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Remmber — after every tolstake return to the beginning of the 
task a^id-take t ml entire group of children (not merely the 
child who aade #4d.8take) through the exercise from the begin* 
nlng, ^either until the children are firm or until they make 
.their next mistake (at^vhich tlae you correct and then return 
to the beginning of the task). (Engelmann & Carnine, 1972, 
p. 14) 

D. Praiae and> leedback ^ 

• Reinforce the children who are on task. Follow the rule of 
^ catching children in the act of beln^ good. Show the mis* 

^ behaving child that he la receiving no revards and that the 

children who m^e working are receiving rewarda. (Engelmann 
> 4 Osbom, 1970, p. 14) 

Always relate the performance of tbe children to the r&lea. 
Do so in a positive manner. *• . • Give the children feedback* 
on each of the behaviors that enter into workli^ hard. This 
medna that you ahould let the children know whch they are 
working hard. *Wafkilig hard' actually covers a variety of 
behaviors: giving the correct response; following your pre* 
sentation — looking at the chalkboard, listening and respond* 
Ing to inatructiona, answering questions. (Science. Research 
Associatea, 1971b, p. 60) ^' 

E. Pacina ^ 

Pace your presentations so that you move rapidly fb the right 
placea but slowly when necessary* Move quickly enough for 
th£ children to see the point of each taak — always at a rate 
that will maintain their interest and enthusiasm. (Engelmann 
& Osbom, 1970, p. 14^ 

♦ 

According to the Die tar curriculum authors (Engelmann and his associates), 
these are the basic Implementation variables. It is assumed that if a 
teacher braves in these ways the children will achieve the academic objec- 
tives of the Distar program. That is»^ the Distar curriculum developers be- 
lieve tha^^the above teacher behaviors are directly related^to student 
achievement. The following quotation from the introduction to the Distar two* 
day orientation*tralning manual indicates this belief: 



14 



13 



■.Si 



You [the teacher]* should learn how to present the tasks 30 that 
even the lowest -per forming children will learn rapidly* Without 
this workshop training, the chances are that you will not teach 
the Xcmst performers in your clfift* With the training, however, 
you sTOuld be able to reach children that you fa^ve not been albe 
to r^ach in €he past. The teaching techniques that you practice 
herj^ will help you become a better teacher of all your children, 
^ but will make the biggest difference with your low-performing 
cnldren (Science jtasaarch Associates, 1971b, p« 2) 

/ # 

Development of an Observation Instrument 

The deyelopMnt of a set of procedures to record the frequency, content, 

and sequence of behaviors in a classroom, ' Jl well as t^o ratf the quality of 

instructional activities, is complex indeed. The purpose of this section Is 

not to discuss the issues and problems o developing observation iillltriaients. 

a 

These are discussed elsewhere (Rosenshine, 1971, 1973; Rosenshlne & Furst, 
1971). Rather, the translation from in-service training manuals and curriculum 
' guide specifications to the development of a cinrriculunr-specif ic observation 
instrument is illustratad for one area of teacher behavior in the Distar 
program — correction proeedures and criterion teaching. These variables were 
selected because (1) they aiM unique to the Distar program and have received 
consistent and statistically significant support (Siegel & Rosenshine, 1973). 
and (2) they are the most difficult behaviors for most Distar teachers to 
implement appropriately. 

The Distar teacher in-service training manual outlines the procedures 
for correcting basic mdl^takes (Science Research Associates, 1971b). All mis- 
takes are divided into three types--illu8trated by the following example: 
The teacher points to a green triangle and asks^ "What shape is this object?*' 
A child may respond, "A square." This type of mistake represents a lack of 



(A,): 

"A^trlangle." 

"Givlxft the answer" (A) Is slnply- tailing ti^ child the corract 
resp; 



^ProvldJ^A additional Infarmatlon" (A.) la not telling the child 
entire answer but merely providing extra Information so that 
child/can ''cone up vlth*' the correct response. 
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Information ; the child confuses a triangle with a square* A second response » 
Illustrating a jaotor or speech problem . Is •'Why-angle. V The child cmnot 
clearly pronounce "trl." A third chlld'^may not understand the 'Signal > He 
responds, "Green." The signal was not '"What color Is thjs object?" The 
child did not understand what he was being asked; he answered a different 
question. p, ^ 

TRe teadier la taught to correct differentially tha^S tliree types of 
mlsiakea: 

I Type I Mistake-^-Lacks Information 

[1. Teacher gives the answer (A) or provides additional information 

\ 



2. Teacjhet^ests the child by repeatV^pg the segment missed (T) : 
' I. '•What shape Is this object?" 

"Testing the child or children" (T) Is asking the question again 
or requiring the child(ren) .to respond. 

Type II m^take — Motor/Speech Problem 

1. Teacher giUs the answer (A): "A triangle." 

2. Teacher i^peats the signal (R) : "What shape is this object?" 

'^Repeating the signal" (R) is behaviorally identical to testing 
the child (T) . The teacher asks the question again or repeats the 
coomand to respond. The difference in labelling is a function of 
intent. Normally, the teacher does not expect the child to correctly 
answer when slfe "repeats the signal." The purpose of this step is 
merely to call attention to what the children should be responding 
to. 



■J \ 

3. Teacher leads the child (L ): "Say it vith me: TRJangle. . -.again. . r 
V TRIangle... once more. . .TRlShgla." "Leaditig" (L^) la responding with • 

^ the child. The teacher and the child tlmnltaneouAly say the response. 

The "n" Indig^tes the number of times the teacher hays the response^ 
; with the child. Ideally, the teacher should 1^ two 6tL more times. 

r4. Teacher t^ts the child by^ repeat lng_the segment missed C^s: •*What ^ 
shape is this object?"- />v / . ^ ' ^ 

Type III Mistake — Does Not Upherstand Slgnal^^ ^ • 

1. Teacher repeats the signal or calls a|fention to t^r^ignal (R) : 
"Listen: What shape is this object? Shape .". ,V 



,2. Teacher or another child models the response (M) : *'Let*8 listen 
to Anthony do this. AfitiMf^ what shape is this object?" "A 
triangle," he responds'. ^ 

**lIodellng" (M) is performing the tAcher's part and the child's 
parte Th44 is done to de^nstrate to the child how the two parts 
are related. The teachet^ asks the question and then answers the 
question. Note that in (giving the an^er (A), the teacher only 
answers the question. / 

! * ^ V r • 

3. Teacher tests the child by repeating the segment mls^ed^.CT): "What 
shape is this object?" ^ 

An eight*point rating scheme was devised. The scale reflects the thrust, 

purpose, and logic of the correctlipn paradigm—namely: 

a. There are two main steps the teacher should follow when correcting' 
basic mistakes: do something that will prevent the mistake from 
occurring again and redo the segment of the task'taht was incorrect. 

b. What the teacher does to prevent the mistake from occurring again 
JjB a function of the type of mistake (see above). 

c. The teacher must always test the child by repeating the segment 
missed. -If the child responds correctly, the teacher may proceed. 
However, if the child responds inappropriately, the teacher must 
again correct the mistake /(provide additional information, lead, 
model, etc). This testing insures the children's mastery perfor* 
mance. k 

d. No mistake should be ignored. '^^f^'". 
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The teacher's handling o£>|pach nlstak'^ Is rated according to the following 
scale: rr* ^ . \ 

8: Xeacher cprrects the tnlfitake iomedlately after It occurs and acopr- 
ding to the above procedtxres. Then the teacher tests the child | 
(or group) by repeating the segment of the task ^hat was missed, f 
(Example for Type I .mistak^r A, T) 

7: Teacher palrk the type of mlstaloe vlth^ the correct protc^dure.as'^ 
Indicated above but adds additional procedures aia^ tes^s. (Exam-: 
pies for Type I mistakes: M, T; M, L^, T; R, A, T) ; 

6: Teacher pairs with type of mistake an incorrect procedure (and/or 
omitting appropriate steps) and t'ests. (Examples for Type I mis- 
takes: .Lj, T; R, L^l T) 

5: Tests only or repeats l^e entire task only (Exipp^es: Ty; E) 

4: Like 8 but Vith no testj< ^ 

' ■ ■ " 

3: Like T;but with no tei^t. r 

"""^ ■ « 

. '2: Like 6 but with no test. — - * '* 

1: Teacher ignores the mistake or gives the wrong answer or Infonnation. 
.Criterion teaching is initljated after a mistake has been corseted. ^The 
teacher returns to the beginning of the task. According to the authors of 
the Ditftar system, this procedure is necessary so that the children learn that 
Mtch step of a task^is related to a goal and certain rules; the steps or ques- 
tions are not independent. F6r example, a ch4.1d makes a mistake in sounding- 
out the word am^ th^^ child says ''an." Implementing the correction procedures, 
the teacher would touch the m and say "iiii i iii i Bii iii." She would then a^k (he child » 
''What sound is this?" The teacher Would then return to the beginning of the 
'^ask**^requiring the children to sound-out the entire word. Criterion teaching 
demonstrates to the children that ehiSt ^^ ^^^s necessary but -not sufficient. 
The goal, in tfiis case, is to read the entire^word — not merely to identify the 
last sound .v^ t * > 
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The ln4>llcatlon for an observation Instrument Is clear. According to the 
program authors there should be a one-to-one correspondence between the number' 



of student mistakes and the number oi^Vbnes the teacher returns to the be- 
ginning of the task. Therefore, atf each oiis^ake Is cosmltted, the observer 
would 8c6re a tally If the teacher repeats the task from the beginning. This - 
count is independent, however, of the teacher's rating for correcting the mis* 
take. For example, the students are instructed to sound-out the word am (steps 

a througli d of the format illustrated ix^f±g)itfi 1) . The children say ^ 

■ ■ ! - ' , . 

as they sound-out* tkg word. To actually read the word, they must 



^leam how to put the soutids tigether say thej^rd at a nor^ial speaking 
rate. Thus, at step e the teacher says, '"Say it faid^" If one or more children 
say the word at a slower than normal^ speaking rate (e.g., aaammm) , an observer 
would code this error as a type III mistake (not understanding the signal). 
The teacher corrects the ^ror and thie observer would code the teacher's 
Correction procedure (assign a rating from one to eight) • Tf the teacher then 
^ returns ^^to the beginning of the task (step a) , the observer would count this 
as an Instance, of critej^ion teachlag. * ^ 

^riess rigorous interpretation of criterion teaching could also be | 
measured: (1) Count the number of times during a session that the teacherX 
repeats a segment of the task but not the entire task (e.g., a mistake occur.- 
ring at; step £ in Figure' 1 and the teacher rettirning step £ rather^^^i^n a); 
(2) count the number of times the teacher recycles through a segment ot the 
task or the complete task but not Itamediately following a mistake (e.g., a 
mistake occurring at step in Figure 1, the teacher proceeding with step f^, 
but theti. returning to step^ or O. 



The observation Instrument used to code teacher behaviors in the Dlstar 
system Is program-^speclf Ic and Includes both rating ^nd counting measures of 



teacher behavior. There are ^ther forms/ of systematic observlit^oti which 
could be used depending on tb^ curriculum ^d the age of the Student's. Vor 
example, questlonnalr.es hkve been developed which allow older^ students to 
rate the' quality and specify the quantltv of certain lnstructi6nal behaviors — 

that the' teacher Is wlsil organized, that^the teacheP^i presentation. Is clear, 

\ / 

or that the teacher r^ponds to student Idetfs. ^thouguoany of these student 
questionnaire are program-general, there ^s been sqds to develop instru- 

ments which focMir^!(^^hQ||||^pecl|^ InatnfllMi'H^^ctlviA^es which are emphaslzecf 




see; for exasple, Kochendorffr , 1966, or 



and An 



Walberg 



Summary orl^failts for Di|(tar Research 

In tW^^studies (Siegel & Rosenshlne, 1973) , It was determined that teacher 
behaviors ' that were considered important for successful program Implemeatatlon 
(following the format, using appropriate correction procedures, teaching a 
format to criterion, requiring unison responding to ^signals) were related to 
studipit achievement* The correlational analysis showed that in a predictive 
sense :^ ' . 
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It is not only important^to attempt to correct mistakes when 
they occjjr, but it is^lso important to correct the mistakes 
accordlilg to the correction paradigm. 



2. It is important that the teacher get unison responses from the 
group. Th8%i^i8, none of the children should be allowed to 
prompt other children's responses. 
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3. Praise for appropriate responding and attending behavior Is 
\ wumpontant.. This, of course, does not mean that It Is 
t xmloportant for things other than achievement, for example, 

httmaneaess or civility or positive self-Image, 

4. It Is Important to follow the fprmat — both fo;^grpup and* 
-Individual tasks. Slight modifications In the format are ' 

^ permissible. 

Furthermore, a later study (Slegel, 1973) provided experimental support 
for specific correction procedures and criterion teaching — two categories ot 
behaviors which are characteristic of the Dl^tar curriculum package (See 
F^ure 2) . Randomly-selected groups of high and ^ow Implementing teachers 
were retrained In techniques of a) correcting students' mistakes according 
to. a^^especlf led procedure attd b) recycling^ through an Instructional task " 
until all of the chlldreici In the group respond without error. As a result 
of retraln'lng, the 23 experimental teachers performed at a significantly 
higher level of Implementation that the 27 "control" teachers (those not 



Insert figure 2 about here 
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retrained). fact, the performance of the ^xp^AAJmCIitaJt tou) Implementors 
was superior to the performance of the CjonX/Lot ktgh Implementors after re- 

training. In addition, there were significant differences In achievement 

\ 

(favoring the experimental group) on the students* pos^-test scores after 
they w^re statistically adjuisted for the differences on the pre-test scores. 
Thus, significant changes In teacher behavior (and particularly along the 

dimension of criterion teaching — the behavior of repeating the entire task 

\ 

after a mistake has been corrected) apparently caused significant changes in 



student achievement (on^»oth a program-specific and program-general criterion- 
referenced measure) . ^ ^ ^ 

Perhaps the most Important^ aspect of these studies was obtaining a 
functional -rel^tionahip 'between^eacher behavior and student achievement in 
a highly-struct^^ cwriculum such as the Distar program. This suggests 
that even In a curriculum program tSiat controls teacher behavior to the* 
extent that it specifies word for word what 'to say to a group of studenta^ 

T ' ^ • ^ 

there rema.lns a large ^unount jerf-^arlation in both teacher b^havioj. and 
stud^ent performance^ This uMer scores the iiq>ortance of studying th^ kinds 
of variation in teacher *behavl^ that produce desired changes in student 
behavior. ' ^ 

Implications for Training Distar TeacheVs 

These studies support the hypothesis /that ticking each format t^ * 
criterion (pr mastery) resilts-vin/superlM achievement performance. .The 
goal of critejrion teaching is that every child in the group will be able to 
respond correctly to every segment of the format, from beglnninsj^ to end, 
without being corrected. And unlesSj^ teacher can effectively correct mis- 



takes, it is unlikely that every child in the^ group can respond correctly to 
every segment of a task withp^ an^^ proiiq)tlng from the teacher. Thus, a tea- 
chert]^ ability to correct mlstakesf is logically related to her ability to teach 
a fozWat to criterion. 



3 

The discussion of measures of ^ student outcomes (achievement, attitudes and 
dispositions to act, and personal development) has not been included here. This 
omission is not Intended to minimize the issues. Two papers by Rosenshine 
(1970^ 1971) are recommended as an introduction to this complex problem. 



the retraining program used in the above study presented a correction 
paradigm which at step, three emphasizes ;»xer ion teaching: I) do something 
that will prevent.; the mistake f rofil^Mcurrlng again ^a different procedure is 
sugges^ted for each type of mlstake«~lack of inf prmation, ^motor or speech pro* 
)>lem» or not understanding Ih^^ignal) ; 2) redo the segment of the task' that 
vas"^ incorrect; and 3) redo the task from the' beginning with fhe entire group*. 
If a child makes a mistake at step two or at' step three, yie teacher returns' 
to step pne. This procedure is continued until all children in Che group can 

r. . 

perform all segments of the format — from beginning ta end-— without error* 
Unfortunately, merely describing and demonstrating a procedure to a 
gfpup »of tea(f^er^ do^ not insxire that the teachers will successfully imple- 
ment the paradigm. As a result of this form of training, teachers can often 
verbalize the correction and criterion teaching procedures but rarely can 
befiave in a manner cpnsistent vlth the paradigm. ; 

An alternative form of training was used successfully in the reported 
study. During the retrainlng^^i^rkshop the teachers progressed through a 
series of exercj|^es which sxiccessively approxln^ted actual teaching situations. 

^The teachers would present in unison a format to the trainer who would act 

4 ' 

the part of a child. At various points fn the task, the trainer would make 
a mistake. THe teachers would then correct the mistake according to the 
paradigm. If one oK^more of the teachers in the group JU/\a}ppKOpujoJtSL)bi QJQKr 
KHQXaA the misfake, the trainer would: 1) explain the proper procedure; 



4 

Unison responding is efficient, provides group support, and requires that 
all teachers participate (as opposed to working in small groups which usually 
leads to off^task behavior). 

23 
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2) make the ^mistake aglin so that the' teacher^ could apptopr lately correct 



It; and* 3) require the teachers tt> return to the begiiiniiVrof the- tas)c*« In 



other words, the trainer would correct the teachers' procedural mistakes in 
the same^way the teacher would- correct children's mistakes. 

A similar procedure was used later when ^e trainer worked with each 

^ ■ • , ■ ^ J ^ • - V . 

teacher in the ciassrbom with her own'.stu^ents. If the te^liejr'did not per- 
fom the. Qcxrrectlon and criterion \teac^li% paradigm upprppriitteiyy the trainer 
would Intertupt the presentation, provide feedback to the teacher, and require 



the teacher to return to t^ beginning of ther'tasky^ Although onq might suspect 

that the teachers would be iq>set by this, each teacher verbally e3q)resse^^ 

enthusiasm for the procedure; and many teachers requested further, training. 

This positive attitude towards training may have been due in part to the 

drAmatic improvements in s^d^t performance, as reported by the teachers, 

during the few days between tfe&v workshop and the classroom y^it. ^ 

These training pr<^cedure8 are nQtjmique« however, Bushell, while 

training teachers to /implement his Behavior Analysis Follow-Through Program, 

used similar procedures: » ^ 

We began with a summer institute. We had clear effects on 
the attitudes of the participants — they loved it. We meak^ured 
to see whether there was any change in what they were doing in 
the. classroom after they kot home and found none. So we abandoned 
sunmer institutes. The other thing we're taught to do as profes- 
sionals is to be consultants— wise men who drop in and tell every- 
body' how to do it right. We^ sent a polished consultant from district 
to district and would take data on some specific aspects of teaching 
and 'learning before the arrival of the consultant and after his 
i' departure. * And without knowing his travel schedule, we couldn't 
\ ^tell\i^en he had been there. So we dropped consultants. 



In one eastern city, the entry point was obtained through sheer 
desperation. We brought everybody together for three days and 
literally stood beside each one of them and said, 'Do it this way.' 



, For some reason It had a huge effect, although there's nothing In 
my background that suggests that that's an appropriate way to 4.0 It/ 
(Maccoby & Zellner, 1970, p. 110) ^ y^,^ 

: ■ " ■ . . • ■ '^y ' ■ 

A General Research Strategy 

- ■ • — / ■ 

The descriptive-correlational **e«pfeflmental loop paradigm Is perhaps 

* . ' • • w ■ ' * 'V ^ - ^ '* ^ 

m^Bt readily understood^lthin the context of a b^havlorally oriented or 
structured ^urrlculum prograrnvsuch as the Dlstar Instructional System. Yet Jt 
It Is proposed thbt this re^earclL^trategy would be applicable to less strup-v 
tured and more '.'^open'' curricula as well. 

Table 1 Illustrates this application and suggests various program- 
specific (Important) process variables and possible outcome' varla|>les for 
tRree early childhood programs . , 

The program Implementation (process) variables and outcdme variables are 
prl)[Barlly determined by the curriculum developers but ma^ or may not be 
stated explicitly. Fuxr|:hermore, the behaviors whlcir^ire emphasized during 



pre-servlce and Inservlce teacher training may vary from what Is expressed 
In the writings (Journal articles, books, teaoiier guides, etc.) of the currl- 
culum designers.' Nevertheless, the researcher and progiMm developer must 
ultimately concur on the Implementation variables and on the procedures and 
Instruments used for collecting teacher and pupil data. If this were not the 
case, then the program developer could argue — just^lflably — that the reseayi:;her' 
study did not test the program's Implementation var4.able8. It would he^'^ther 
difficult for a researcher to justify the ^Inclusion 6f a part^ular "pro- 
gram-specific" variable on an observation Instrument when this claim Is denied 
by the program authors. A compromise solution, however. Is possible. The^ 
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Table 1 



Examples of Process and Oytcome Variables for Tliree Early Childhood Education Programs 



Program 



Example Df Process Variables 



Example of Outcome Variables 



The Ehgelraann-Becker 
Program 

(Engelmann and Becker, 
sponsors) 



Following the format for group and individual 
activities; 

Correcting all mistakes according to a pre- 
speeified paradigm; 

Teaching all formats to criterion (mastery); 
Requiring unison responses to signals. 



Reading, language, and arithmetic 
achievemeilt measures—criterlon- 
referenced and norm-referenced. 



The liSgnitively Or- 
iented Approach 
(yelkart, sponsor) 



ro^t 



Arranging the envirotfent so that^ child 
can actively interact with it; ~ 
Allowing the child to establish the learning 
pace; 

Arranging the curriculum so that it progresses 
from the simple to the complex— from the 
concrete to the abstract. ' 



Rating scales to measure effects on 
socioeconomic development; 
Measures of language development; 

and ^ 
Academic achievement measures. 



I 



EDO Program 
(Armington, 
sponsor) 



Teacher flexibility; Ability to pose questions; Measures to assess student initiative 
Helping child to verbalize and to formjate his and self-direction, curiosity, and ^ 



ideas and thoughts; 

Emotional climate established by the teacher; 
Arranging the environment for corrective feed- 
back; Allowing the child to engage and disen- 
gage from a task at the child's will. 



imagination, openness to change, 
self-respect and respect for others; 
Concentration, listening and remembering. 



Note: The description of variables for each program was inferred from the literature; program authors for each 
curriculum may disagree with the wording or emphasis in this table. The purpose here Is to illustrate the 
versatility of the detcriptlve-correlatlonal-experimental loop^'fTesearch paradigm and not necessarily to give 
an accurate description of process and outcome variables for specific programs. 
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The observation InstruiMtit cotild reflect three types of variables: those 
variables vfalch the curriculum developers and researchers hypothesize to be 
laportant for the success of the prograa, those variables trtilch only the 
curriculum diivelopers hypothesize to be Important, and those variables lAlch 
only the researchers hypothesize to be Important. The research would then 
reflect each group's biases. This proce\|ure could be eiq>anded to Include Isqile- 
mentation variaoles which are specific or important to other curricula, as 
well as variables (especially outcome variables) which are important to dif- 
ferent groups (parents, educators, legislators, students, etc.). For example^ 
Armlngton would be concerned with measures of curiosity and imagination for • 
children in the EDC Program whreeas many parents may be concerned with measures 
of reading and arithmetic achievement. Both sets of variables could' be 
collected — not only for the EDC Program but fo^, other programs as well. 

The research paradigm, furthermore, does not narrowly specify the inanner in 
which implementation andfoutcome variables are to be collected. Observation 
instruments cotQd be used — or possibly student ratings, teacher or parent 
questionnaires, video taplngs, audlotapings, surveys, or a series of "unob- 
trusive" m^ures. In fact, certain programs will typically value one form 
of dlta conectlon over anotlier. For example/ the Engelmann^Becke 
%rould be satisfied with norm-referenced and criterion-referenced tests of 
achievement. The EDC, Program, on the other hand, would probably value more 
indirect measures of student behavior and attitude. Again, many types of 
measures could be used for each program if more generallzable relationships 
are of Interest. 
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lapllcat Ions' for Teacher Education 
A aajor concern iiaplied in the introduction to this chapter Is whether 
or not generalized teaching behaviors are of Inportance to the liapleBentatlo^ 
of curriculum progran< It was suggested that teacher behaviors vhldi are 
specific to a pro-am nay be nore ln^ortant in influencing student outcones 
than teacher behavio rs w hich apply to a vide range of progrmaa. In dee d , no , 
teacher teaches the curriculum '^first grade reading;" she teaches the SWBIT 
Reading Program, the Bank Street Beaders, IPI, Addlson-Wesley, Dlstar Beadd^g,^^ 
McGraw-Hill, SRA Reading Labs, or another reading program. Teacher behaikgrs 
which are critical to the success of one program may not be very Important to 
the succees of^^^tnothev^^-ov \ay even be detrimental. 

^I^df there axi§ similarities among programs. For examplV, a teacher 
surely would not have to learn a new set of teaching skills when she teaches 
Dlstar Reading after having taught Dlstar L a n guage. Likewise, there are 
certain similarities among programs oriented toward cognitive growllk, between 
curricula oriented toward behavior modification, and so on. However, it is 
hypothesized that as the classification becomes more encomp^slng — goes be-* 
yond a specific curriculum program to include programs of a general type and 
ultimately, models o^ Inst ruction— the probability decAeo^eA that any teacher 



Furthermore, some teachers teach no Identificble reading program. Rather 
it is a mixture of (he text^(or several programs), experience, and casual 
practice. In this case, tt^e Issues become more complex because of these 
ambiguous interactions • 
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behavior thabsjju^ies to all programs^of the larger set will be a powerful 
variable (that is, will accoimt for a large percentage of the variance In the 
outcome measures) for all programs of one or more subsets, ^ y 

Pre-servlce Education ^ I ^ 

- \' 

Teacher training programs which' emphasize general strategies for teaching 

4 * 

may be providing Information for the school teacher. \Alch is less than useful. 
Rather, would*-be teachers could profit more from learning ^to teach a sample of 
program types and the behavlpra crucial to each. For example', a student^ 
teacher Interested In early childhood education could learn to implement three 
or four^currlculixm programs which saiq>le a wide range of instructional straite* 
gles: say, Gllkeson and Zimlles* Bank Street Program, Bu8hell*s Behavior 
Analysis Program, Gordon's Florida Project, and WelRart's Cognitively Oriented 
Approach. Or rather, -if a more specialized training were desired, the set 
would include only programs of a certain type: for example, Engelmann's Distar 
Program; B^ishellVs Behavior Analysis Program, and , Res nick* s Primary Education 
Project. Thus, teacher behaviors which arc specific or Importactt to a parti- 
cular program may be learned as well as those implementation behaviors or 



Instructional activities which are generally important- for the success of 
several programs,^ 



No matter what the criteria chosen for the grouping and classification of 
programs — a^, grade level, subject matter, mode of teaching, psychological 
orientation, etc.~it is hypothesized that this holds true. 
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In-service Education 



In-*servlce training should abandon teacher Institutes vfalch focus on 
general lnstructlonal%cClvltleSy methodologies, philosophies » or practices. . 
Instead » the training should emphasize ^ose competencies or bfhavlors vfalch 
are necessary to implement the curriculum programs that the teachers use in 
the classroom.. 

This Implies new roles and responsibilities for curriculum publishers and 
authors as veil as for school supervisory personnel: 

1. The program developers must clearly specify those Instructional 
activities ^^ilch are critical to the l^>lementation of the curriculum. 
Vague prescriptions will lllcely result in vide variation in teacher iaple^ 



mentation. ^ j 

2. School administrators must demand that publishing houses provide * 



adequate training programs. Even well-vrltteh teacher guides are often 
Inadequate training tools. 

3. The school administrators must also provide a sufficient amount of 

time for the teachers to receive the ^available training. 

( ... 

4. Teacher supervisors (curriculum facilitators) must monitoj the 



degree to which the teachers are appropriately impl^ementing the curriculum 
materials package. The curriculum facilitator must review the skills taught 
at training sessions, prompt critical behaviors In the teacher, aid the 
teacher in learning to change his behavior, assist administrative personnel 
evaluate the teacher, and provide information about skills which need prac- 
tice during in-service workshops. 
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^ 5* And finally yth€ curriculum authors and publishers must demonstrate 
that there Is a fuxi^ctional relationship between' the teacher Implementation 
behaviors and student outcomes of Interest. It Is unfortunate that most 
school acfanlnlstrators do not^emand this evidence before committing sub- 
stantial fi^s for curriculum 'materials* 

We are now beginning to recognize that simply developing a 
currlculixm materials package instructional method, or an 
educational innovation is' not sufficient; we are now beginning 
to^ recogaize that studying the way an educational product is 
used in the schools la at least as iiiq;)ortant as developing the 
product* But we have spent too little time and money studying 
how, products are used and modifying products on the basis of 
' such study. (Rosenshine, 1971, p* 70) 
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Figure CaptLana 

Figure 1. Examplji of a format. In* the Dl^ar Reading I prograai. I 
■ ' tR) ' ' ^ ^' '^^ -' ■ 

(From Dista*^ Reading I, Second Edition, Teacber Presentation Book A by 

Siegfried JBngelmann and Elaine C. BrunerV^ Copyriglit 1974, 1969, Science 

Research Associates, Inc. Reprinted by permission of the^ publisher «) 

Figure 2. Scope and sequence of the descriptive-correlatlonal--exper: 

mental study for the Dlstar Language progtram (SlegeH 1974)0 
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am 



TASK 13 CHILDREN SOUND OUT THE WORD AND SAY IT FAST 



a. 



e. 
f . 



Voa'/te going to Kzad tiuj^ lOoKi. You' At going to 60und it out 

and 6aif iZ iqu^t. 
Touch the ball for am. Vm going to ioUxM tht OJUUM and 
touch thz Aoundt. Whtn J touch tht liut 6ound, you. ioy it. 

Keep on 6ai/ing iZ untit I touch thz ntxt hound. 

Von*t'6top betuieen tht houndA. 
Sound it out. Get Kzady, Move to a. aaa. Vhaa aaa la fin, 
ttova quickly to &• The children are to aay aaaMB 
^rlthout pausing between the sounds • 
Return to the ball. Agoft RepeA c until firm. 



When aaam is firm, say: may iZ iajt^t. (Signal.) i^. 
What wnd did you Kzadf (Sttial.) Ve^, <un. GocdHzjoding. 



TASK 14 INDIVIDUAL TEST 



Call on different children to sound out the word in task 13 and 

say It fast. 
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